Prediction with Missing Inputs

For the purposes of this paper, data mining involves: Predictive modeling (Estimating parameters is not of major interest) Nonnormal data with nonlinear relationships Large data sets This paper will not cover issues regarding small data sets, such as Bayesian predictive distributions, or tree-based models, for which specialized methods are available for handling missing data. Missing data are a major concern in data mining, both because a substantial proportion of the data may be missing and because predictions must be made for cases with missing inputs. The terminology used in data mining comes from a variety of fields, including statistics, artificial intelligence, and engineering. The terms training, estimation, and model fitting will be used interchangeably. A target is what statisticians call a dependent variable, and an input is an independent variable. Generalization is the ability of a model to make good predictions for data not used during training. The main purpose of data mining is to get good generalization. The purpose of this paper is to see how to get good generalization with missing inputs. Main points: What’s true for linear regression with multivariate normal data often is not true for nonlinear models (such as neural nets) with nonnormal data. What’s true for estimation often is not true for prediction.